System for analyzing interactions and reporting analytic results to human operated and system interfaces in real time

ABSTRACT

A computerized system for advising one communicant in electronic communication between two or more communicants has apparatus monitoring and recording interaction between the communicants, software executing from a machine-readable medium and providing analytics, the software functions including rendering speech into text, and analyzing the rendered text for topics, performing communicant verification, and detecting changes in communicant emotion. Advice is offered to the one communicant during the interaction, based on results of the analytics.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention is in the field of telephony communications andpertains particularly to methods and apparatus for analyzing telephonyinteractions and reporting the results of analysis in real time.

2. Discussion of the State of the Art

Call centers are organizations or companies that represent one or morecompanies providing goods and/or services to a consumer base. Moststate-of-art call centers are dual capable meaning that they may handleboth switched telephony calls and data network telephony (DNT) calls. Acall center typically includes a plurality of workstations comprisingcomputers connected to a local area network (LAN) or equivalent network.Each workstation includes a telephone or equivalent for handling voicecalls. Voice calls may also be handled directly n the LAN-connectedcomputers using appropriate telephony software.

Call centers are equipped generally to handle a wide range ofinteractions including live voice including voice over Internet protocol(VoIP), connection oriented switched telephony (COST) and messagingincluding email, instant messaging, and chat. Other forms ofcommunication may also be supported like video conferencing, faxcorrespondence, and network-based applications for collaborating. It isimportant to a call center that all call center activity is recorded andmade available for data mining, training, and other processes that relyon historical data for improvement of the performance of the center.Software adapted to report call center activity are typically part of acall center software suite.

Existing call center reporting packages provide information on callssorted by called party number or calling party number, and may includeinformation gathered by an interactive voice response system. A problemwith such reporting packages is that they often lack information on thecontent of the conversations that occurred between callers and callcenter agents whom are trained to handle call center interactions. Suchinformation, if it is present at all, is in a separate set of reportsand not linked to the core set of call center reports. The call centeradministrator thus lacks a comprehensive view the activity in thecenter.

Another limitation in call center technology is the lack of useableinteraction analytics that can be provided to agents and supervisors inreal time to help them optimize the interaction experience of thecustomer and agent. Current call center systems do not provide speechanalytic information to the agent or supervising personnel for example,but are limited to more static screen pop information that is eitherfurnished by the caller at the start of the call or retrieved from adatabase during the interaction between a caller and an agent. Existingsystems cannot advise the agent based on information gathered andanalyzed about the current conversation.

Therefore, what is clearly needed is system and methods for capturingand analyzing current interaction activity including speech and textinteraction such that the outcome of analysis can be made part of thehistorical record of the call center and be made available to managers,supervisors, agents and systems in such a way as to enable correctiveintervention to occur during the interaction process.

SUMMARY OF THE INVENTION

A problem stated above is that it is desired in a call centerenvironment to be able to manipulate the outcome of an interaction inprogress using intelligence developed during the progress of theinteraction. However, current means for acquiring the intelligenceneeded to determine how best to handle the interaction is limited totapping into information already known about the caller or toinformation provided by the caller before or during the interaction as aresult of some prompting made to the caller.

While real-time behavioral analysis of the interaction process ispracticed in some systems to help predict when an upsell should beoffered or when an invitation to a live interaction should be providedto a customer engaged in a non-live interaction with a system or a Website, these systems known to the inventors fall short of the ability toprovide a whole analysis of the current behavioral state of the callerwith respect to emotional state and topical interest of the caller thatmight be exhibited only during the interaction in progress with thecaller or party to a non-live or live interaction.

Moreover, in known systems where some intelligence is provided byanalyzing browsing behavior, typed text, or voice inflection or tone,the intelligence is not developed in a sufficient timeframe to be usedto manipulate the outcome of that interaction at the time of theinteraction or requires human intervention to determine if amanipulation is required and what action should be pursued.

The inventor therefore searched the art of call center data reportingsystems, speech recognition systems, and interaction routing systemslooking for elements that exhibited a propensity for integration in anautomated environment. Every call in center is driven by the success ofagents and automated systems relative the process of engaging andretaining customers to produce higher numbers of sales and moreprofitable sales. One byproduct of this activity is the existence anumber of customers who have negative experiences, failed to completetransactions, or otherwise are dissatisfied with tier experiences duringthe interaction process.

The inventors realized during an intuitive moment that if during theinteractive experience, a more complete analysis of call center customerpreferences and emotional state could be performed and if suchintelligence gleaned could be automatically leveraged to determineappropriate actions that could influence the outcome of suchinteractions in a positive way through a semi-automated or whollyautomated process, then higher numbers of sales and more profitablesales would result.

The inventors therefore constructed a unique system for monitoring andanalyzing interactions that enabled automated development of a morecomplete salvo of intelligence used, in some cases in automated fashionto effect an influence in the outcome of some interactions in real timethat otherwise would suffer some negative result.

Accordingly, in an embodiment of the present invention, a computerizedsystem for advising one communicant in electronic communication betweentwo or more communicants is provided, comprising apparatus monitoringand recording interaction between the communicants, software executingfrom a machine-readable medium and providing analytics, the softwareincluding rendering speech into text, and analyzing the rendered textfor topics, performing communicant verification, and detecting changesin communicant emotion. Advice is offered to the one communicant duringthe interaction, based on results of the analytics.

In one embodiment the electronic communication is voice communicationand occurs over a telephony network integrated with a computer networkand wherein the two or more communicants include a call center agent anda call center customer or customers. Also in one embodiment theelectronic communication is a voice communication and the speechinteraction is analyzed for speaker verification, and emotional state ofone or more communicants. In another embodiment the electroniccommunication is a text communication and the text interaction isanalyzed for communicant verification and emotional state of one or morecommunicants.

In one embodiment the system performs emotion detection by phraseinterpretation, character state interpretation and, or by emotion iconrecognition. Also in one embodiment the apparatus provides call controland media processing. Further in one embodiment the advice offered is apre-defined upsell script based on prevailing topic of thecommunication, the script served to one of the communicants in realtime.

In one embodiment the pre-defined script is selected from a pool ofpre-defined scripts and served to a GUI operated by the communicant, thecommunicant a call center agent. Call control may be performed by asession initiation protocol (SIP) server. Also the media session markuplanguage (MSML) may be used to control the voice platform supporting thevoice communication. Further the media resource control protocol (MRCP)may be used to control the software analytics. In some cases theanalytics for topic recognition may include configurable modes for hotword recognition and for continuous speech recognition.

In another aspect of the invention a method for advising one communicantin electronic communication between two or more communicants isprovided, the method comprising the steps of (a) monitoring andrecording electronic communication between the communicants; (b)executing software from a machine-readable medium, providing analyticsincluding rendering speech into text, analyzing the rendered text fortopics, performing speaker verification, and detecting changes inspeaker emotion; and (c) offering advice to the one communicant duringthe interaction, based on results of the analytics.

In one embodiment of this method in step (a) the electroniccommunication is voice communication and occurs over a telephony networkintegrated with a computer network and wherein the two or morecommunicants include a call center agent and a call center customer orcustomers. Also in one embodiment in step (a) the electroniccommunication is a voice communication and in step (b) the speechinteraction is analyzed for speaker verification, prevailing topics, andemotional state of one or more communicants.

In some embodiments the electronic communication is a text communicationand the text interaction is analyzed for communicant verification,prevailing topics, and emotional state of one or more communicants. Alsoin some embodiments the emotional state of the one or more communicantsmay be achieved by phrase interpretation, character state interpretationand, or by emotion icon recognition. In some cases in step (c) theadvice offered is a pre-defined upsell script based on prevailing topicof the communication, the script served to one of the communicants inreal time. In still other cases the pre-defined script is selected froma pool of pre-defined scripts and served to a GUI operated by thecommunicant, the communicant a call center agent.

BRIEF DESCRIPTION OF THE DRAWING FIGURES

FIG. 1 is an architectural overview of a call center practicing realtime interaction analysis and real time reporting of results accordingto an embodiment of the present invention.

FIG. 2 is an architectural overview of a call center practicing callervoice analysis and script service according to an embodiment of thepresent invention.

FIG. 3 is a block diagram illustrating basic components of a speechanalysis and real time reporting system according to an embodiment ofthe present invention.

FIG. 4 is an exemplary screen shot of an agent scripting interfaceaccording to an embodiment of the present invention.

FIG. 5 is an exemplary screen shot of a graphical user interface forviewing call center activity in real time according to an embodiment ofthe present invention.

FIG. 6 is an exemplary screen shot of a script and rules configurationinterface according to an embodiment of the present invention.

FIG. 7 is a process flow chart illustrating steps for analyzing voiceduring a call in real time and determining a script for an agent basedon the results of analysis.

FIG. 8 is a process flow chart illustrating steps for interactionanalysis and historical reporting according to an embodiment of thepresent invention.

DETAILED DESCRIPTION

The inventors provide a system for monitoring and analyzing aninteraction and for reporting the results of analysis of theinteraction. The system incorporates one or more interaction analysisengines enabling continuous voice and text recognition with topicdetection, speaker verification, and emotion detection. The system ofthe present invention is described in enabling detail using thefollowing examples, which may represent more than one embodiment of theinvention.

FIG. 1 is an architectural overview of a call center 103 practicing realtime interaction analysis and real time reporting of results accordingto an embodiment of the present invention. A communications network isillustrated in this example and includes call center 103, a publicswitched telephone network (PSTN) 101, and a wide area network (WAN)102.

WAN 102 may be a corporate or private WAN including but not limiting tothe Internet network. In this example WAN 102 is an Internet network andmay be referred to hereinafter in this specification as Internet 102.PSTN 101 may include wireless carriers and other telephony networksconnected thereto by bridging and is meant to encompass any telephonenetwork including digital and analog networks. The inventor chooses theInternet and PSTN as accessible networks to call center 103 because ofthe high public access characteristics of those networks.

Call center 103 is dual capable meaning that it may receive interactionsfrom PSTN 101 and from Internet 102. Call center 102 has a local areanetwork (LAN) 112 provided therein and adapted for transfer controlprotocol over Internet protocol (TCP/IP) among other known Internetprotocols, not mentioned here but generally known to the skilledartisan. LAN 112 supports various personnel workstations as well as ahost of nodes adapted to facilitate practice of the present invention.LAN 112 supports a plurality of agent workstations illustrated herein asagent workstations 113 (1-n). In this example, each agent workstation113 (1-n) includes a LAN-connected personal computer (PC) and atelephone. Agents operating at stations 113 (1-n) are required to loginto the call center system and may be located by the system using adirectory server such as a directory server (DIR) 124 connected to LAN112.

LAN 112 supports at least one manager or supervisor workstation (MGR)116. Workstation 116 also includes a LAN-connected PC and a telephoneconnected to PBX 109 by internal telephone wiring 110. Other equipmentmay be included within a workstation of an agent or manager withoutdeparting from the spirit and scope of the present invention. Theinventor deems that illustration of a LAN-connected PC and associatedtelephone is sufficient for explaining the various aspects of thepresent invention.

In this example call center 103 has a centralized telephone switch 109through which calls are received at the call center and placed from thecall center (outbound). Switch 109 may be any type of call center switchincluding an automatic call distributor (ACD), a soft switch(implemented in software), or a private branch exchange (PBX). Switch109 is a PBX in this example. Switch 109 in call center 103 hasconnection to a local network switch 107 by way of a telephone trunk108. In this example the local network switch is also a PBX switch.Callers placing telephone calls into call center 103 or receivingtelephone calls from call center 103 are illustrated herein as telephoneicons 104 each telephone icon representing a potential customer of callcenter 103.

A customer station 105 is illustrated within PSTN 101 in this example.Customer station 105 has a telephone analogous to telephones 104connected to local switch 107 via network telephone wiring. In oneembodiment cellular telephones including smart telephones are used inplace of landline telephones. Customer station 105 includes a personalcomputer illustrated herein as connected to the Internet networkdirectly or through an Internet service provider (ISP not illustrated).Other more direct methods of Internet access are known in the artincluding digital subscriber line (DSL) technology, integrated digitalservices network (ISDN), broadband, wireless fidelity (WiFi), cablemodem, etc.

All calls arriving at switch 109 in call center 103 are ultimatelyhandled by the call center through live assistance from an agent orthrough the assistance of some automated system. PBX 109 is enhanced inthis example by a computer telephony integrated (CTI) processor 114,also supporting an instance of interactive voice response (IVR) unit.Processor 114 is connected to switch 109 by a CTI link 117. Voice callsfrom Internet 102 and from sub-telephony networks connected to PSTN 101may all arrive at switch 109 as incoming telephone voice calls. Agenttelephones illustrated in agent stations 113 (1-n) are all connected byinternal telephony wiring to PBX 109.

LAN 112 supports an Internet protocol router (IPR) 115 within callcenter 103. IPR 115 is adapted to provide direct Internet access to LAN112 in a continuous manner such as on a 24/7 basis. IPR 115 enablesbi-directional data and communication flow between call center agentsand systems and resources including customers connected to the Internetnetwork. IPR 115 has connection to Internet 102, more particularly to an

Internet backbone illustrated herein as a double arrow extending throughcloud 103. The illustrated backbone logically represents all of thelines, equipment and access points that make up the Internet as a whole.Therefore there are no geographic limitations to the practice of thepresent invention.

A chat server (CHS) 121 is illustrated within the domain of Internet102. CHS 121 has a digital medium accessible thereto for storing andserving data and for storing and executing applications enabling serverfunction. CHS 121 is adapted to enable live chat sessions between agentsfrom call center 103 and any potential customers having access to thenetwork using a computing appliance capable of network navigation. Onewith skill in the art will recognize that a computing appliance capableof accessing Internet 102 may include but is not limited to a desktopcomputer, a laptop computer, a personal digital assistant or a smartcellular telephone.

A Web server (WS) 122 is illustrated within the domain of Internet 102.WS 122 has a digital medium accessible thereto for storing and servingdata and for storing and executing applications enabling serverfunction. Web server 122 is an electronic information server adapted toserve Web pages in hypertext transfer markup language (HTML) or anotherpresentation protocol such as wireless application protocol (WAP). WS122 may host one or more Web sites of one or more companies representedby call center 103. WS 122 may be though of as a customer access pointto call center 122. One or more call center applications may beimplemented in Web services description language (WDSL) or other knownapplication protocols generally known to the skilled artisan includingthe inventors. A Web page served by WS 122 to any potential customer mayinclude various options for making contact to call center 103 includingvoice-based, and text-based interaction through call buttons, chatoptions, and messaging options.

Internet 102 has an email server (ES) 123 illustrated therein. ES 123has a digital medium accessible thereto for storing and serving data andfor storing and executing applications enabling server function. ES 123is adapted to serve email to agents operating at agent workstations 113(1-n) through IPR router 115. In one embodiment LAN 112 supports ageneral email router (EMR) 125. EMR 125 is a special email routingservice for determining final internal routing to agents (over LAN 112)for emails sent to a generalized email address for the call center.

A universal routing server (URS) 111 is illustrated within call center103 and connected to LAN 112. URS 111 has a digital medium accessiblethereto for storing and serving data and for storing and executingapplications enabling server function. URS 111 is adapted to serveinternal routing strategies to requesting nodes over LAN 112. Initialrouting strategy is typically invoked through a caller pre-screeningprocess conducted using the IVR running on CTI processor 114. URS 111may also be used to determine routing strategy in the case of a redirector subsequent routing requirement as a result of interaction processing.

Voice calls registered at switch 109 may be intercepted by the IVRrunning on CTI processor 114 to determine the purpose of the call forthe benefit of selecting an internal routing destination for the call.The present invention includes a method for monitoring those calls thatare routed to a live agent for the purpose of ensuring quality of theinteraction and for gleaning information from the interaction in realtime whereby that information is used in one or more analysis routinesto provide additional data for historical reporting in one embodiment.

In one embodiment of the present invention, a historical reportingserver (HRS) 118 is provided within call center 103 and connected to LAN112. HRS 118 has a digital medium accessible thereto for storing andserving data and for storing and executing applications enabling serverfunction. HRS 118 is adapted to record all call center activity andserving the information upon request in the form of specialized reportsordered from an interface such as a graphical user interface software(GUI SW) 127 illustrated on the computer of MGR station 116. As wasdescribed further above in the background section of this specification,historical reporting at the time of filing this application is limitedto mechanical call data such as average handling time (AHT), results ofthe call, and any caller information that was already known to thesystem or that was obtained in caller pre-processing. Informationconcerning the caller's mood, and topics of preference that arise duringthe course of the interaction and validity of the caller's identity arenot available in standard reports.

A call control media server (CCMS) 119 is provided within call center103 and connected to LAN 112. CCMS 119 has a digital medium accessiblethereto for storing and serving data and for storing and executingapplications enabling server function. CCMS is integration between acall control server and a media-processing server. CCMS 119 is adaptedto set up and tear down the call legs of a call and the media processoris adapted to process any associated media such as voice, video, andother media applications. In this example, HRS 118 and CCMS 119 areconnected together via a separate digital network. CCMS is alsoconnected directly to CTI processor 114 via the separate data network. Acall center database (CCDB) 120 is provided and is made accessible toCCMS 119 and to HRS 118. Monitoring occurs within CCMS of all of theinteractions represented therein. In one embodiment, CCMS records alldata to CCDB 120 and HRS 118 serves all requested data from CCDB 120.

CCMS has at least one analytic tool 126 implemented in software (SW)that is executed at server 119 during interactions in process. Analytictools enable performance analysis of an interaction from the perspectiveof the caller, the agent, or both caller and agent. In one embodiment aspeech recognition system with a topic detection feature is used todetermine what topics customers are talking about or are interested inbased on the call transcripts. The ASR (not illustrated here) may havemore than one mode of operating relative to topic detection. In onemode, the ASR system is set to recognize any hot words that are spokenby the caller. In another embodiment entire conversations are recordedand analyzed for topic detection. A version of the topic detectionfeature may be adapted for processing text-based interactions as well.In continuous recognition mode, the ASR recognizes the entireconversation and can generate a list of topic preferences that wereprominent during the conversation.

In one embodiment the system further includes an emotion detectionsystem for determining if a caller or an agent or both is angry,disappointed, frustrated, etc. The emotion detection system (notillustrated here) may work by analyzing voice inflection, tone, volume,etc. A version of the emotion detection system may be used to analyzetext message where emotion icons are used and where other emotionbearing indications are present like use of all caps in a message orother signs of mood used in a text message.

In one embodiment of the present invention all of the informationdetected during an interaction whether the interaction is voice-based ortext-based is stored with all of the traditional historical informationrelated to the interaction and interaction initiator (customer). Amanager operating GUI SW 127 from MGR station 116 can order and receivespecialized historical reports that exhibit what topics the customersare talking about. The manager or supervisor can get a good indicationof the levels of frustration of callers over a period of time. Theinformation may be presented in any number of customized reports thatcan include information about just a single interaction to statisticsrelative to some of or all of the interactions that occurred and weremonitored by CCMS 119.

Calls received at PBX 109 for internal routing are setup by CCMS 119 inthis example after CTI notification of calls pending. CCMS may utilizeIVR records of the calls to determine with the aid of URS 111 the bestrouting strategies for the calls being set up in the system. URS returnsthe routing strategy for each call based on information sent to it inthe request from CCMS 119, which includes IVR records of thepre-screening process. URS 111 may also subscribe to call centerstatistics on agent availability, estimated wait time (EWT) in queue,and other states before returning a viable routing strategy for eachcall pending. Manual routing and redirection may also be practiced byagents and supervisors logged into the system in place of or in additionto URS participation.

In one embodiment the information gathered during the interactions isused to supplement historical reporting of call center activity addingnew dimensions to what can be learned about the activity. In anotherembodiment the information gathered during the course of interaction isutilized in real time to optimize the interactive experience of thecustomer by providing solutions that overcome perceived states of thecaller concerning mood and topical preferences. In this embodiment theinvention is further characterized by provision of a real-time scriptingservice that may be used to send pre-defined scripts to agents engagedin a live interaction the scripts relative to information gatheredduring the course of the interaction.

FIG. 2 is an architectural overview of a call center 203 practicingcaller voice analysis and script service according to an embodiment ofthe present invention. A communications network 200 includes call center203 and PSTN 101. Internet access for call center 203 is not illustratedin this example but may be assumed present. In this example, the focusis about agent scripting as a result of analysis of information gatheredduring the course of live voice interactions wherein the scripts aredelivered to the agents in time to influence the outcome of thoseinteractions.

Callers 104 represent customers of call center 203, in this example. Thecallers access center 203 through local PBX 107 over trunk 108 andregister at PBX 109 for internal routing. IVR treatment is not precludedin this example. All callers may first be treated via IVR beforedetermining a final destination for each caller. In this example a callcontrol server is provided separately from a media server. A sessioninitiation protocol (SIP) server 201 is provided within call center 203and connected to LAN 112. SIP server 201 serves as the call controlserver in this example. SIP server 201 has a digital medium accessiblethereto for storing and serving data and for storing and executingapplications enabling server function. SIP server 201 uses SIP to set upand tear down all voice calls in this example. Other standard protocolsused to set up and break down voice calls may be used in place of SIPsuch as Call Control extensible markup language (CCXML) for example.

SIP server 201 has a separate digital network connection to an openmedia server (OMS) 202. OMS 202 has a digital medium accessible theretofor storing and serving data and for storing and executing applicationsenabling server function. OMS 202 is adapted to perform the mediaprocessing part of the interactions set up between live agents andcustomers of the call center. In this example OMS 202 may be called bySIP server 201 on an as needed basis.

OMS 202 includes a recording (REC) engine for recording voiceinteractions monitored in SIP 201. Analytic SW tools 126 for determiningtopic preferences and moods or emotions exhibited by callers and perhapsagents as well are provided on OMS 202 instead of in an integrated callcontrol/media server analogous to CCMS 119 of FIG. 1. A data repositorylabeled database 203 is provided as a centrally located and shareddatabase for SIP server 201, OMS server 202, and HRS server 118.

A supervisor station 206 is illustrated in this example and may beanalogous somewhat, at least in physical implementation, as MGR station116 described with reference to FIG. 1. Supervisor station 206 has a GUISW 205 provided thereon and executable there from. GUI SW 205 includesat least one enhancement over GUI SW 127, which is the capability ofallowing the supervisor to engineer agent scripts off line and to usecurrent information about a call in progress to send particular scriptsto the agent to read to the customer during the course of theinteraction.

In one embodiment a script server (SS) 207 is provided within callcenter 203 and connected to LAN 112. SS 207 has a digital mediumaccessible thereto for storing and serving data and for storing andexecuting applications enabling server function. SS 207 is adapted tosend designated pre-defined scripts to selected agents whom are inprogress with a call that the script is intended to influence theoutcome of. A data repository 209 labeled scripts is provided andconnected to SS 207 by data link. Repository 209 is adapted to store allof the relevant scripts that can be selected and served to agentsworking calls. In one embodiment a supervisor or other authorizedindividual such as a supervisor operating station 206 has a hand incausing a script to be selected from repository 209 and served to anagent. In another embodiment scripting service is entirely automaticwith the scripts being selected by a software program after analyzingresults of a voice call in progress.

An agent script interface (ASI) 208 is provided as a graphical userinterface on each agent station such as agent station 113 (3) asillustrated in this example. Every participating agent working the livecalls may be assumed to have an open instance of ASI 208 running on thedisplay screen of their LAN-connected computers. ASI 208 is adapted toreceive and display scripts sent from SS 207. ASI 208 is not absolutelyrequired in order to practice the present invention. A script can beengineered to execute and display on its own as an alert script thatappears in a window. ASI 208 provides a more convenient and efficientway to display scripts and the scripts that are displayed may be savedand recorded along with the interaction data in the historical record ofcall center activity.

To illustrate an example, consider a caller registered at switch 109.The routing determination to route the call to agent 113 (3) is madeafter IVR pre-screening. Agent 113 (3) picks up the call and beginsservicing the caller. In the meantime recording engine 204 is recordingboth the agent and the caller. An ASR system is turned on with topicdetection and hot word recognition. An emotion detection system isturned on for the caller's voice but not for the agent's voice. Duringthe course of the call the detection tools determine that the hot wordtrade was recognized and the prevailing topic is upgrading to a newsystem with the trade. Also during the course of the call the caller'smood or emotional state is determined to be excited or very positive.

The original call may have been a service call for an existing systemthe caller has but with the newly detected information, a monitoringsupervisor operating at supervisor station 206 aided by GUI 205 hasdetermined that a scripting service is appropriate for the currentsituation with the immediate interaction being evaluated. The supervisormay highlight the caller and select an action such as “send scripting toagent of this call”. The information gathered during the call is usedalong with one or more rules to determine which script to send from apool of scripts. The decision to select and send a specific script to aparticular agent involved in a call is made automatically once thesupervisor invokes the service on behalf of a call in progress.

SS 207 selects the correct script based on information in hand and rulesgoverning which script to send based on that information. In this caseSS 207 selects a script from scripts database 209 and serves it to ASI208 open on the PC display screen. The script may appear with a systemprompt urging the agent to read the script to the customer. In this usecase the script may be an offer to “upgrade” to the next best system andto accept the “trade” offered by the customer. In this case emotiondetection determined that the caller was excited reinforcing the idea ofintervening with an up-sell script. In this embodiment all of theactivity including the caller and agent voice recordings, transcripts,and the topic prevalence, and the emotional state of the caller and/oragent is recorded in the historical record of activity at the callcenter and is subject to inclusion in one or more custom reports orderedby a manger or supervisor.

In this embodiment scripting may be ordered for any agent that iscurrently involved in a current interaction. Selection of scripts isdependant in part on what emotional state exists with the caller andwhat topics are prevailing in the conversation. In one embodiment thesame topic and emotion detection schemes can be used to influence a livechat or an instant message session. In one embodiment Web visitors canbe offered services base on analysis of the Web behavior that can belinked to topic prevalence or to an emotional state. For example, atopic can be detected based on the visitor's page preference orclick-history at the Web site. The emotional state of the visitor mightbe deduced by observing the navigation behavior, for example going backand forth repeated on the same sequence of Web pages could indicatefrustration.

FIG. 3 is a block diagram illustrating basic components of a speechanalysis and real time reporting system according to an embodiment ofthe present invention. Open Media Server (OMS) 202 from FIG. 2 isillustrated in this example in more detail. Components of analytic tools126 of FIGS. 1 and 2 are in one embodiment hosted on OMS 202. In anotherembodiment they may be hosted within the call control server such as SIPserver 201 described in FIG. 2.

An automated speech recognition (ASR) system 308 is provided in thisexample and has two separate modes of speech recognition. ASR system 308has a hot word recognition mode (HWRM) and a continuous speechrecognition mode (CSRM). ASR system 308 has a connection to an ASRlibrary repository 305 adapted to store vocabulary and other datarequired to enable ASR function. In this embodiment ASR system runs onOMS 202, which is connected by a data link 314 to a monitoring CTIprocessor and therefore has access to the live voices of the agent andthe customer of any particular call being monitored.

Output from ASR system 308 may be recorded by a call recording systemillustrated herein as a call recording system 315. Output from ASRsystem 308 may be processed for topic detection by a topic detectionsystem (SYS) 310. Topic detection system 310 may be turned on or off ina pre-configuration process. With topic detection turned on, the systemattempt to determine if one or more high priority topics are prevalentin the conversation between an agent and a customer of a voice call.Topic detection system 310 may parse transcripts of the conversation todetect topics that are primary in the conversation.

Output from ASR system 308 may be processed for emotion detection by anemotion detection system (SYS) illustrated herein as an emotiondetection system 309. Emotion detection system may work in conjunctionwith speech recognition to detect if any prevalent emotional states canbe detected in either the customer's voice patterns or in the agent'svoice patterns, or both. Emotional detection system 309 may analyzevoice inflection, tone, volume, pattern and other voice characteristicsto determine if any significant emotional state can be implied to thecustomer, to the agent, or to both the customer and the agent during aconversation. Emotional detection SYS 309 may be turned on or off duringa pre-configuration process.

Output from ASR system 308 may be processed by a speaker verificationsystem 311. Speaker verification system may detect if there are delaysor inconsistent answers to specific questions typically asked of callersto verify that the caller is who they say they are. If a caller delaysor fails to answer one or more of these types of security questions, thespeaker verification system may generate an alert and send the alert tothe agent and/or supervisor to alert them of a possibility of fraud.Speaker verification system 311 can be turned on or off during apre-configuration process. In one embodiment speaker verification system311 has access to a repository containing speaker verification data suchas answers to security questions that are kept on file. In oneembodiment speaker verification system 311 may also have access to athird party validation system that may provide some validation servicefor callers. In one embodiment voice patterns of repeat callers arerecorded and saved. In this embodiment the speaker verification systemmay compare a callers voice against a sample of the caller's voice kepton file for the purpose.

In one embodiment ASR system 308 is capable of receiving and deliveringrouting instructions to an agent engaged in a conversation in a casewhere some analysis of a customer's speech warrants additional routingto another party, for example, to diffuse a crisis or to help pacify adisgruntled customer. In this case ASR system 308 may have connection toa universal routing server (URS) 306. URS 306 is analogous to URS 111described further above with reference to FIG. 1.

In this example, OMS 202 includes an interaction queue 307 that isadapted to represent current interactions being processed by the systemof the invention including an indication of which parties to eachconversation are being analyzed by one or more of the analyticprocesses. In a visual representation of queue 307 each conversation hastwo parties listed, an agent (row labeled A) and a customer (row labeledC). Reading from left to right, a first interaction represented in queueindicates that the customer will be analyzed but the agent will not. Thesecond and third conversation queued indicate that both parties to theconversations will be analyzed. The pattern follows for every columnrepresenting a two party conversation. In one embodiment the queuerepresentation may be extended for three party calls and conferencecalls where some of the participants may be analyzed and some may not.

The granularity of configuration for agents may come down to each agentfor configuring if and what analysis routines will be performed on thatagent's voice when in conversation with a customer. In most embodimentsall customers calling in have their speech processed for emotion andtopic prevalence. However, customers may be configured individually aswell. For example, a rule could be created for John Doe to alwaysanalyze his voice for emotional state when he calls in. It may be a factthat John is easily upset and often expresses disappointment or angereven though he may be a great customer. In one embodiment queue 307contains current interactions in progress where an agent and a customerare actively engaged. In another embodiment queue 307 containsinteractions that are still pending but where the agent that will takethe call has already been selected by the routing server.

Output from call recording SYS 315 is sent to a real time reportingserver (RTRS) 301. RTRS 301 has a digital medium accessible thereto forstoring and serving data and for storing and executing applicationsenabling server function. RTRS 301 is analogous to HRS 118 describedpreviously with reference to FIG. 1. RTRS is connected to historicaldatabase 112 introduced and described previously with reference toFIG. 1. All recordings made of conversations are saved as well astranscripts of the conversations and other media processed with thoseconversations.

In one embodiment where it is desired to influence the outcome of aconversation, output from emotion detection system 309 and from topicdetection system 310 is input into a speech analysis recommendationengine (SARS) 302. SARS 302 is adapted to take the results of emotiondetection system 309 and the results of topic detection system 310 andprocess those results using one or more algorithms to determine anappropriate course of action given the results of speech analysis. SARS302 has a digital medium accessible thereto for storing and serving dataand for storing and executing applications enabling server function.

SARS 302 may be connected to a rules base (not illustrated) containingrules pertinent to what courses of action should be taken based on whatinput parameters. One course of action is a rerouting of the caller to anew destination. Another course of action is serving a notice to agentscontaining suggested scripting for the agent to read to the customer.Any negative indication found through agent voice processing may resultin a disciplinary recommendation, a temporary re-assignment, a simplewarning, or a request to attend a specific training session, or thelike. SARS is, in one embodiment connected directly to RTRS 301 via adata link. In this case SARS 302 sends its output to RTRS 301 forhistorical data storage and for real time reporting.

In one embodiment SARS 302 may have a LAN connection for the purpose ofcommunication directly with agent stations and supervisor stations.However in this embodiment SARS 302 only reports to RTRS 301. RTRS 301is adapted to server data relative to current conversations to GUI 205running in this example on the computer of a supervisor workstation 304.It is noted herein that voice analysis data and recommendation data mayalso be made part of the historical record of call center activity.

Data served from RTRS 301 to GUI 205 for display may be packaged in theform of one or more custom reports 313. In one embodiment a supervisorsubscribes to RTRS 301 and may pre-configure rules relative to whatsolutions may be recommended by SARS 302. The supervisor may alsopre-configure the types of reports that are desired and for who in thegroup including customers and agents engaged in conversation the reportswill be about. For example, a supervisor may order a report that revealsa real time percentage of the total number of current customers engagedin monitored conversation that have had a negative emotions ratingduring their conversation.

In one embodiment of the invention, a supervisor working at supervisorstation 304 may select and serve scripts to agents based on real timeinformation about their current conversations with customers. Agentworkstation 303 includes agent-scripting interface (ASI) 208 and isadapted to receive scripting sent by a system or by a supervisor ormanager. RTRS 301 may send a report to the supervisor of workstation304, the report displaying in GUI 205. The report may alert thesupervisor to favorable topical preferences in some of the currentconversations in progress. The report may include a list of one or morethan one recommended script (recommended by SARS 302). When SRARS 302makes a recommendation, that recommendation is almost immediatelyavailable to a supervising entity through push alert or through receiptof a custom report that contains the recommendation. In one embodimentGUI 205 includes a real time activity panel that when displayed listsall of the current interactions being monitored, identifies the partiesto those interactions, and displays any real time analytic resultsfleshed out during those interactions such as current emotional statesand topic prevalence states.

FIG. 4 is an exemplary screen shot 400 included in an agent-scriptinginterface according to an embodiment of the present invention. Screenshot 400 represents a system recommendation that includes scriptingservice to an agent that is currently interacting with a customer.Screen shot 400 includes a title bar 401 that identifies the screen as acall recommendation. Screen shot 400 may be referred to hereinafter as acall recommendation screen 400. Title bar 401 of call recommendationscreen 400 is personalized to a call center agent John who is therecipient of the recommendation. Bar 401 identifies the caller currentlyin interaction with agent John as Mr. Smith. The title bar alsoindicates the time in for the current interaction, in this case 1: 20seconds meaning that John has been on the telephone with Mr. Smith forone minute and 20 seconds. The time status feature on the title barincrements accordingly as the call progresses. A link between the ASIand the call control server enables the stopwatch feature for everyconversation.

In one embodiment a caller identification number 402 is displayed in thecall recommendation, which could be the Mr. Smith's telephone number.Real time analytic results 403 may be displayed in the callrecommendation screen. In this case a hot topic or hot word is “SystemUpgrade”. Therefore, the customer has mentioned or asked about a systemupgrade either directly or casually during the interaction. The callrecommendation screen may include a prompt 404 intended for John. Prompt404 simply asks John to read the closing script 405. The system hasalready determined that the call will be routed to anothermore-experienced agent so John must end the conversation until a newagent picks up the call. In one embodiment the script displays forreading and becomes part of the interaction history between John and thecustomer Mr. Smith. The script simply informs Mr. Smith that he will berouted to another agent Ben that will help him with upgrading hiscurrent system. Agent John will likely hold to make sure that the otheragent is in place to accept the call.

A system prompt or command 407 is displayed beneath closing script 405to transfer the caller to Ben Leech ID # 30124. The system has alreadydetermined that Ben Leech is the best possible match for the caller atthe current time. Call recommendation 400 includes transfer options 406of automatic transfer (system performed) or manual transfer (agenttask). While agent John is still on the telephone with Mr. Smith withthe transfer in progress, Ben Leech picks up the call and reads anintroductory script 408 sent to his ASI and to the ASI of John.

In one embodiment script 408 is not a pre-defined script and may be anactual text rendition of what Ben says when he picks up the telephonewith John and Mr. Smith on the line. After the greeting from Ben, asystem prompt 409 appears on call recommendation 400 asking John toplease hang up the telephone and take the next call. A script server mayserve call recommendation 400, or it may be served by a speech analysisrecommendation system (SARS) analogous to SARS 302 of FIG. 3, or by asupervisor operating a supervisor station like station 304 of FIG. 3. Inanother embodiment the script appears in the call recommendation window,which is generic to the agent-scripting interface. In this case bothscripts and system prompts may appear as required in the interface usingthe screen as a template the data appearing in the appropriate fields.

FIG. 5 is an exemplary screen shot 500 of a graphical user interface forviewing call center activity in real time according to an embodiment ofthe present invention. Screen shot 500 represents a current call centeractivity view or report that may be ordered by or pushed to a supervisorin charge of a particular group of agents. This view may be served tothe supervisor GUI analogous to GUI 205 described previously. View 500has a title bar that includes the title CC Activity View [Real Time].The title bar also has the current time and date. The view ispersonalized to a Mr. Blake, which is the recipient of the view.

View 500 may include a window dedicated to the display of groupstatistics 501 as they are at view time. In this case Mr. Blake is asupervisor in charge of a campaign utilizing a specific group of callcenter agents. Reading from top down the data reveals an Agent Groupassigned to and scheduled to work the campaign. The Agent Group=Group(C) [System Sales]. In this case the type of campaign is an outboundcalling campaign. The campaign duration=6 hours. There are a total of 22agents working the campaign. An interactive option [List} may enable thesupervisor to call up a detailed interactive list of those agents.

Real time statistics indicate that there are 19 calls in the queue and 8calls in progress, meaning 8 calls are currently being monitored andprocessed for information using the voice analytic tools describedabove. The estimated wait time (EWT) in queue is 4 minutes while theaverage handling time (AHT) for a call is 3 minutes. The current closingrate is 56% meaning 56% of the total number of calls resulted in a sale.

View 500 includes emotion detection system [ED-SYS Data] 502 for thegroup of interactions that are currently in progress. Of the 8 calls inprogress, there are 2 calls where the caller is angry. A telephone icondisplays each call having an angry caller and an indication of how longthe call has been in progress. The freshest call with an angry caller isjust 1.00 minute old. The next call with an angry caller is 2 minutesand 43 seconds old. With an average handling time of 3 minutes, thesupervisor may elect to intervene only on the freshest call at 1:00minute. Each of the call icons is interactive and can be manipulated tocall up all of the information about the call, the caller, and theagent. An interactive option labeled Send Script is provided to theright of each troubled conversation. A supervisor may click on theoption to call up a manual script selection interface. The supervisormay first highlight the conversation that she wants to influence andthen may browse for or select a script from a list or pool ofpre-defined scripts. Once the script is selected, hitting sendautomatically causes the script to be sent to the ASI of the agent inthat conversation.

Also listed in parameters 502 is the total number of calls in the groupof 8 calls in progress where the agent is angry. In this example only 1call is in progress where the ED SYS has detected anger in the agent. Atelephone icon represents that call. Next to the icon an indication isprovided of the current age of the call, in this case 2 minutes and 10seconds. A script or other intervention may be warranted for that call.Another set of real-time parameters 503 is provided below the ED-SYSData. These data are Topic Detection system data [TD-SYS Data]. Of the 8calls in progress, 5 of those have hot topics detected that areprevalent in the conversation. The 5 calls are listed in order fromfreshest call to the oldest call. In one embodiment the supervisor maymouse over any one of the displayed call icons to get the most prevalenttopic of that call. The hot topic for the oldest call on the list is“Upgrade”. Some conversations may be listed in more than one data set.On conversation may have a hot topic detected, an angry caller detectedand an angry agent detected.

Caller verification alert bar 506 informs the supervisor of how many ofthe current conversations have callers who are suspect of fraud. In thisexample the number is 0 calls. This number should typically remain at 0or very low with good security practices. A set of interactive options505 is provided in view 500 beneath the standard parameters. Forexample, an option to add or subtract agents from the group (C) ispresented. A supervisor may also be able to transfer agents from onegroup, say group C to another group say group D. A next interactiveoption allows a supervisor to perform a manual rerouting of a callerfrom one agent to another agent. This process may also be whollyautomated using a script server. A next option enables the supervisor tosend an up sell script to any agent involved in conversations.

View 500 has some pre-configuration options 504 that a supervisor mayset prior to running a call campaign. A first option is let the systemdetermine the best actions to pursue in the face of caller anger, toptopic, agent anger, etc. In this case all of the routing and calltransfer steps are automated. Another option is prompt me and ask mewhat to do. In this case the system only take corrective action at thebidding of the supervisor. The last option enables the supervisor toinitiate all of the actions. In this case the supervisor selects andserves scripts and performs rerouting from the desktop. In a case wherethere may be hundreds of active calls automation may be the best rulewhereas for small groups a human operator may manage the calls inprogress.

FIG. 6 is an exemplary screen shot of a script and rules configurationinterface 600 according to an embodiment of the present invention.Configuration interface 600 may be a static part of the supervisor GUIfor receiving reports. The interface has a title bar defining the windowas a Script Configuration Interface. Interface 600 enables an agent toconfigure rules and to write scripts that may be selected for scriptingservice to agents in the field. The interface includes a visual list ofavailable topics to write scripts about. The available topics are hotwords that if spoken by a caller could cause the system to recommend ascript. In interactive option 602 is provided to enable the supervisorto add or remove topical hot words. The hot words would be recognizedduring automated speech recognition performed by the ASR system. Hotwords may also be parsed from text-based interactions in some embodimentof the present invention.

Script configuration interface 600 includes access to available scriptsthat have been pre-configured and stored in a repository analogous tothe script repository 209 described with reference to FIG. 2. In thisexample, the supervisor has access to five available up sell scripts,two available pacification scripts, and two available transitionscripts. An up sell script is an offer to upgrade or to purchase ahigher priced product or product version. A pacification script might beused to calm an angry customer. A transition script is used to inform acustomer that he or she will be redirected to a different agent tofinish the transaction.

An interactive option 604 is provided within interface 600 to enablecreating and adding new scripts to the pool of existing scripts. Aninteractive option 605 is provided that enables a supervisor to add oredit existing scripts. An interactive option 606 is provided thatenables the supervisor to delete existing scripts from the pool ofscripts. Scripts are selected by rule based on results of analysis ofcustomer and/or agent behavior, topic detection and emotion detection. Aspeech analysis recommendation engine (SARS) described further above maybe responsible for processing those results before a recommendation ismade. A recommendation to script an agent may be passed to thesupervisor interface (GUI 205) and the supervisor may have the lastdecision of whether to send a recommended script or to send a differentscript or no script. In one embodiment script selection and service iscompletely automated and occurs in the background without human input.

Interface 600 includes a rules configuration pane 608. There are severalinteractive options made available to a supervisor through rulesconfiguration pane 608. An interactive option is provided that enablesthe supervisor to create a new rule. An interactive option is providedthat enables the supervisor to edit an existing rule. A rule created tohelp decide whether to recommend a script or not depends on certainconditions that are inferred through voice analysis and text analysis(live text interaction). An interactive option is provided within rulesconfiguration pane that allows the supervisor to view and edit existingconditions that might lead to rule application and subsequent selectionof one or more scripts. In one embodiment more than one script might berecommended whereby the supervisor picks one of the list to send to acustomer.

Configuration pane 608 also contains an interactive option forspecifying new conditions for triggering rules. The exact conditionsspecified will depend on the nature of the campaign. An interactiveoption is provided within rules configuration pane for linking new rulesto existing or newly created scripts. Interaction with any of thedescribed options in configuration pane 608 may bring up new windows tofacilitate the tasks specified by the option. An interactive option forsaving and submitting a particular configuration of rules and scripts isprovided as well as a link that enables testing a new configuration todetermine if the results and rules applied in a situation lead torecommendation of the appropriate scripts or script.

FIG. 7 is a process flow chart illustrating steps 700 for analyzingvoice during a call in real time and determining a script for an agentbased on the results of analysis. At step 701 a participating agentpicks up a call from queue. The call may have been previously processedby IVR to determine internal routing strategy. At step 702 the systemturns on the ASR system for the call. The call may or may not berecorded by default. In one embodiment all calls are recorded. In oneembodiment the call recording system is turned on for each call that thesystem determines should be recorded.

At step 702 the system may select an ADR mode such as continuous voicerecognition or hot word recognition. A feature of the ASR system forrecognizing topics may be turned on or off by default. Typically topicdetection is on by default for the purpose of generating systemrecommendations. The system may turn on the caller verification systemat step 703. In one embodiment step 703 occurs only after the ASR systemrecognizes one or more security questions asked of the caller by theagent. The system may turn on the emotion detection system at step 704.

All of the features of the ASR system including topic and emotiondetection may be run for the caller's voice only or for both the callerand the agent. In one embodiment certain agents will be monitored whilecertain other agents in the group may not be monitored. Typically allcallers would be monitored but it is possible to selectively monitorcalls while other calls are not monitored. At step 705 the systemattempts to verify the callers identity. In one embodiment the ASRsystem recognizes a security question and then listens with emotiondetection and topic detection turned on to determine if the question isanswered correctly and if so was it an uncomfortable answer for thecaller. For example, is the security question was “What is your date ofbirth” and the caller has a hard time responding even though the rightanswer is given, the system may make a determination whether the calleris who the caller claims to be at step 706.

If the caller verification system determines that the calleridentification cannot be confirmed, then at step 707 the call may beterminated and a fraud alert may be sent to the managing supervisor atstep 708. There may be several differing schemes for verifying theidentity of a caller including matching voice pattern of the caller witha voice pattern of the caller that is kept on file at the call center.Generally if the caller successfully answers one or more securityquestions without much hesitation the caller ID is confirmed. The systemof the invention is compatible with most security regimens such as areused by financial institutions and the like.

If the caller verification system confirms the identity of the caller atstep 706, then as the call is being handled, the system checks if anyprevalent topics can be discerned for the ensuing conversation at thecurrent time in step 709. If no hot words or topics are identified bythe system in step 709, the system may determine subsequently or at thesame moment if there is any significant emotional state detected in theconversation. For example, the caller may become frustrated or the agentmay become frustrated or crass. If no topics could be verified and noadverse emotional states are detected, then the process may skip to step714 where the agent finishes the call without any outside influence.

It is noted herein that the steps of topic and emotion detection may becontinually running during the life of the conversation and eithersystem will take action if triggered by topic recognition or by therecognition of an emotional state or by both. It is also noted thatdetection of a prevalent topic or topics in a conversation may beconfirmed while no significant emotional state can be determined.Likewise, an emotional state may be determined without topic recognitionbeing successful for a conversation.

At step 709 if the system detects a prevalent topic for theconversation, the system classifies the topic and at step 710 and mayalso select a script for the agent to read at step 711. In oneembodiment if a topic is detected that might lead to script service tothe agent, the system waits until the emotional state of the callerand/or agent is confirmed if any. In one embodiment a topic may bedetected and a script selected based on the topic but before a script issent, the emotion detection system determines that the caller is angry.A script for an angry caller may be more important than a scriptoffering an up sell to the caller and therefore may take priority in anygiven conversation. After step 709 regardless of whether a topic wasfound or not, the system moves to step 712 for emotion detection unlessthat feature is purposely turned off for the call.

If topic prevalence for the conversation is determined at step 709 thenat step 710 the topic is classified and the agent of the call isnotified to receive a script at step 711 as part of the systemrecommendation described further above with reference to FIG. 3. If thetopic detection system confirms a topic the system will still check foremotional states of the caller at step 712. At step 712 if emotion isnot detected and no topic is detected at step 709, then the process mayskip to step 714 where the agent finishes the call as mentioned above.

If a significant emotional state is detected in the conversation at step712 the system may classify the emotion and may select a script for theagent at step 713. If an emotional state is detected and classified, thesystem may wait to see if a topic could be detected. For example, theemotional state detected in the caller's voice may be one of excitementand the topic that is prevalent might be “new system”, in which case thescript triggered by the topic, perhaps and up sell script, is validatedas well be the emotional state of the caller.

If two different scripts are selected for an agent in a conversation,one relative to topic and one relative to emotion, the system may enablethe agent to make a choice of which script to actually read to thecaller. In one embodiment the supervisor may make the final decisionover which of two widely variant scripts will be read to the caller. Instill another embodiment, an automated routing may weigh the selectedscripts against all of the rules and make a determination of whichscript is most important like a tiebreaker routine. Obviously an angrycaller would not be a good candidate to be read an up sell script basedon a prevalent topic. Emotional states may range from anger,frustration, confusion, and melancholy to happy, excited, and ecstatic.

After the recommended script is sent to the agent at step 711,ultimately the system will save all of the call data including all ofthe results in a historical database for reporting. It is noted thatreal time reporting may include reporting of both real time informationor states and static information that is the same every report likestandard customer data. In one aspect a supervisor may screen a customreport to see only the real-time data. In one embodiment a supervisormay elect to physically monitor all connected conversations from adashboard style interface such as may be accessed using GUI 127 or GUI205. In this case the supervisor may choose which calls to monitor andwhich calls to let go. In one aspect the supervisor may configure topicdetection and emotion detection to automatically turn on for aconversation that exceeds the AHT for a call or some other time limitimposed on the conversation. There are many possibilities.

FIG. 8 is a process flow chart illustrating steps 800 for interactionanalysis and historical reporting according to an embodiment of thepresent invention. At step 801 the system selects an interaction toprocess. Typically, in an open media queue the interaction may be avoice-based interaction (call) or a text-based interaction request(Chat, IM). At step 802 the system determines if the interaction is avoice interaction or a text interaction. At step 802 if the interactionis a voice interaction then the system turn on the ASR system with topicrecognition at step 804. The system also turns on the callerverification system at step 806 and the emotion detection system at step807.

If the interaction of step 801 is determined a text interaction at step802 then the process moves to a process branch beginning with step 803for turning on a text recognition system with topic detection. Then thesystem turn on a text-based emotion detection system, which looks forhints of emotional state within the typed text of a text interaction.

For the voice-based branch of process steps 800 the system attempts toverify the ID of the caller at step 811. At step 813 the callerverification systems decides if the caller is authentic. If the calleris judged not to be authentic at step 811, then the process moves tostep 815 where the call may be terminated. In that case a notice offraud may be sent to the managing supervisor in charge of the agentgroup at step 817. If the caller is determined to be authentic at step813 then the call is monitored for topic and emotion detection and theoutputs of those systems may be combined and aggregated for the call atstep 808.

For the text-based branch of process 800 the outputs of the text-basedversions of the topic detection and emotion detection systems arecombined and aggregated for the text interaction at step 808. At step809 all of the interaction data and results of voice/text analysis arestored in the historical database managed by a real time reporting toolor server analogous to server 301 of FIG. 3. At step 810, the reportingserver may serve the interaction data including results of analysisalong with standard data to an authorized requestor such as a managingsupervisor. Each interaction in progress may be viewed in real timethrough a GUI dashboard style interface for both text-based andvoice-based interactions. In one embodiment the supervisor or managermay select and send scripts to agents in real time based on analyticresults whether the interaction is a voice call, a chat session or aninstant message session in progress. Moreover, the historical record issupplemented with group statistics about topic preference and emotionalstates of interaction participants. These stats may be mined for usefulinformation that may aid in refining scripts, processes, voiceapplications, and other call center processes and objects.

It will be apparent to one with skill in the art that the interactionparticipant monitoring and reporting system of the invention may beprovided using some or all of the mentioned features and componentswithout departing from the spirit and scope of the present invention. Itwill also be apparent to the skilled artisan that the embodimentsdescribed above are specific examples of a single broader inventionwhich may have greater scope than any of the singular descriptionstaught. There may be many alterations made in the descriptions withoutdeparting from the spirit and scope of the present invention.

What is claimed is:
 1. A computerized system for advising onecommunicant in electronic communication between two or morecommunicants, comprising: apparatus monitoring and recording interactionbetween the communicants; software executing from a machine-readablemedium and providing analytics, including: rendering speech into text,and analyzing the rendered text for topics; performing communicantverification; and detecting changes in communicant emotion; whereinadvice is offered to the one communicant during the interaction, basedon results of the analytics.
 2. The system of claim 1 wherein theelectronic communication is voice communication and occurs over atelephony network integrated with a computer network and wherein the twoor more communicants include a call center agent and a call centercustomer or customers.
 3. The system of claim 1 wherein the electroniccommunication is a voice communication and the speech interaction isanalyzed for speaker verification, and emotional state of one or morecommunicants.
 4. The system of claim 1 wherein the electroniccommunication is a text communication and the text interaction isanalyzed for communicant verification and emotional state of one or morecommunicants.
 5. The system of claim 4 wherein the system performsemotion detection by phrase interpretation, character stateinterpretation and, or by emotion icon recognition.
 6. The system ofclaim 1 wherein the apparatus provides call control and mediaprocessing.
 7. The system of claim 1 wherein the advice offered is apre-defined upsell script based on prevailing topic of thecommunication, the script served to one of the communicants in realtime.
 8. The system of claim 6 wherein the pre-defined script isselected from a pool of pre-defined scripts and served to a GUI operatedby the communicant, the communicant a call center agent.
 9. The systemof claim 6 wherein the call control is performed by a session initiationprotocol (SIP) server.
 10. The system of claim 2 wherein the mediasession markup language (MSML) is used to control the voice platformsupporting the voice communication.
 11. The system of claim 2 whereinmedia resource control protocol (MRCP) is used to control the softwareanalytics.
 12. The system of claim 2 wherein the analytics for topicrecognition include configurable modes for hot word recognition and forcontinuous speech recognition.
 13. A method for advising one communicantin electronic communication between two or more communicants, the methodcomprising the steps of: (a) monitoring and recording electroniccommunication between the communicants; (b) executing software from amachine-readable medium, providing analytics including rendering speechinto text, analyzing the rendered text for topics, performing speakerverification, and detecting changes in speaker emotion; and (c) offeringadvice to the one communicant during the interaction, based on resultsof the analytics.
 14. The method of claim 13 wherein in step (a) theelectronic communication is voice communication and occurs over atelephony network integrated with a computer network and wherein the twoor more communicants include a call center agent and a call centercustomer or customers.
 15. The method of claim 13 wherein in step (a)the electronic communication is a voice communication and in step (b)the speech interaction is analyzed for speaker verification, prevailingtopics, and emotional state of one or more communicants.
 16. The methodof claim 13 wherein the electronic communication is a text communicationand the text interaction is analyzed for communicant verification,prevailing topics, and emotional state of one or more communicants. 17.The method of claim 16 wherein the emotional state of the one or morecommunicants is achieved by phrase interpretation, character stateinterpretation and, or by emotion icon recognition.
 18. The method ofclaim 13 wherein in step (c) the advice offered is a pre-defined upsellscript based on prevailing topic of the communication, the script servedto one of the communicants in real time.
 19. The method of claim 18wherein the pre-defined script is selected from a pool of pre-definedscripts and served to a GUI operated by the communicant, the communicanta call center agent.